Design : aes, ibex, jpeg

Platform: sky130hd, asap7

A sample dataset with multiple runs varying the design utilization and the layer_adjust parameters for the routing layers across multiple designs (aes, ibex, jpeg) and multiple platforms (sky130hd, asap7). The same layer_adjust parameter is applied to all the routing layers. All design metrics from the runs are collected for analysis.

Metrics data is represented as json files in the METRICS2.1 format. Each experiment in the run is a separate json file. All of the files are read into a DataFrame 'json_df'.

• Rows of the DataFrame represent an experiment.
• Columns of the DataFrame represent the metrics.

Number of runs in the dataset: 6426
Number of metrics in each data: 89

• The columns in the DataFrame, which are the metrics colleced for each run.

Index(['run.flow__generate__date', 'run.flow__metrics__version',
       'run.flow__openroad__version', 'run.flow__openroad__commit',
       'run.flow__scripts__commit', 'run.flow__uuid', 'run.flow__design',
       'run.flow__platform', 'run.flow__platform__commit', 'run.flow__variant',
       'synth.design__instance__stdcell__count',
       'synth.design__instance__stdcell__area', 'synth.runtime__total',
       'synth.cpu__total', 'synth.mem__peak', 'constraints.clocks__count',
       'constraints.clocks__details', 'floorplan.timing__setup__tns',
       'floorplan.timing__setup__ws',
       'floorplan.design__instance__stdcell__area',
       'floorplan.design__instance__design__util',
       'floorplan.design__io__count',
       'floorplan.design__instance__macros__count', 'floorplan.runtime__total',
       'floorplan.cpu__total', 'floorplan.mem__peak',
       'globalplace.route__wirelength__estimated',
       'globalplace.timing__setup__tns', 'globalplace.timing__setup__ws',
       'globalplace.runtime__total', 'globalplace.cpu__total',
       'globalplace.mem__peak', 'placeopt.timing__setup__tns',
       'placeopt.timing__setup__ws', 'placeopt.design__instance__design__area',
       'placeopt.design__instance__design__util',
       'placeopt.design__instance__stdcell__count', 'placeopt.runtime__total',
       'placeopt.cpu__total', 'placeopt.mem__peak',
       'detailedplace.timing__setup__tns', 'detailedplace.timing__setup__ws',
       'detailedplace.design__instance__displacement__total',
       'detailedplace.design__instance__displacement__mean',
       'detailedplace.desgin__instance__displacement__max',
       'detailedplace.route__wirelength__estimated',
       'detailedplace.runtime__total', 'detailedplace.cpu__total',
       'detailedplace.mem__peak', 'cts.clock__latency__min',
       'cts.clock__latency__max', 'cts.clock__skew__worst',
       'cts.timing__setup__tns__prerepair', 'cts.timing__setup__ws__prerepair',
       'cts.timing__setup__tns__postrepair',
       'cts.timing__setup__ws__postrepair', 'cts.timing__setup__tns',
       'cts.timing__setup__ws', 'cts.design__instance__hold_buffer__count',
       'globalroute.clock__latency__min', 'globalroute.clock__latency__max',
       'globalroute.clock__skew__worst', 'globalroute.timing__setup__tns',
       'globalroute.timing__setup__ws', 'globalroute.timing__clock__slack',
       'globalroute.timing__clock__period', 'globalroute.runtime__total',
       'globalroute.cpu__total', 'globalroute.mem__peak',
       'detailedroute.route__wirelength', 'detailedroute.route__via__count',
       'detailedroute.route__drc_errors__count',
       'detailedroute.runtime__total', 'detailedroute.cpu__total',
       'detailedroute.mem__peak', 'finish.clock__latency__min',
       'finish.clock__latency__max', 'finish.clock__skew__worst',
       'finish.timing__setup__tns', 'finish.timing__setup__ws',
       'finish.power__internal__total', 'finish.power__switch__total',
       'finish.power__leakage__total', 'finish.power__total',
       'finish.design__instance__area', 'finish.design__instance__utilization',
       'finish.runtime__total', 'finish.cpu__total', 'finish.mem__peak'],
      dtype='object')

Data Engineering

• Extract only the relevant columns -- for this exercise, get the design name, platform, flow variant and route metrics including runtime and memory footprint.
• Parse the variant string and extract the relevant features -- Utilization, layer adjust for the routing layers.
• Rename columns.
• Create new columns (features) based on other column values -- for e.g. Success/Fail for the router.

• Print the head of the data frame. As we can see, all of the parameter values used in the experiment is in the "Variant" string. We will have to parse the string and create individual columns for the features we are interested in. For this experiment the only values changing for each run is the Core Utilization and the Layer Adjust. So we extract those and store and column entries in the data frame

	Design	Platform	Variant	GR WNS	Wire Length	Vias	DRC Errors	Route Runtime	Peak Mem
0	ibex	asap7	UTIL_15-AR_1.0-GAP_10-PD_LB_ADD_0-LAYER_ADJUST_0.5-GR_OVERFLOW_0	4427.65	179202	211145	0	22:11.15	7.01316e+06
0	ibex	asap7	UTIL_45-AR_1.0-GAP_10-PD_LB_ADD_0-LAYER_ADJUST_0.5-GR_OVERFLOW_0	4502.23	117639	218727	ERR	4:22:54	5.54974e+07
0	ibex	asap7	UTIL_22-AR_1.0-GAP_10-PD_LB_ADD_0-LAYER_ADJUST_0.6-GR_OVERFLOW_0	4450	156414	220298	0	26:03.42	7.7621e+06
0	ibex	asap7	UTIL_54-AR_1.0-GAP_10-PD_LB_ADD_0-LAYER_ADJUST_0.32-GR_OVERFLOW_0	4510.35	114516	228853	ERR	1:37:06	5.56105e+07
0	jpeg	asap7	UTIL_29-AR_1.0-GAP_10-PD_LB_ADD_0-LAYER_ADJUST_0.42-GR_OVERFLOW_0	2560.06	205054	421858	0	1:11:55	1.48011e+07

• We can see from the above plot that asap7 platform had a lot more successes than sky130hd for the same parameter settings

• In the OpenROAD flow, layer_adjust is used to derate the routing supply of global route, so as to provide more relaxed guides to the detailed router. Using higher layer_adjust values makes the global router more pessimistic and could result in over congestion during global route, whereas using lower layer_adjust values could make the global router more aggressive and creates guides for the detailed router that can cause DRC error or increased run times during detailed route.

• Lets examine the number of flase positives by the Global router, i.e. global router completes but detailed router fails

asap7    428
Name: Platform, dtype: int64

• As we can see from the above data all of the false positives are in asap7

• Extract individual features from the "Variant" string. For this experiment we are interested in only the Core Utilization and the Layer Adjust parameters
• Print the head of the dataframe after extracting the features and performing further data engineering. This data frame is now ready to be used. Notice that we have also created additional columns for capturing whether Global Route and Detailed Route completed.

	Design	Platform	Variant	GR WNS	Wire Length	Vias	DRC Errors	Route Runtime	Peak Mem	GR Success	Success	Core Util	Layer Adjust
0	ibex	asap7	UTIL_15-AR_1.0-GAP_10-PD_LB_ADD_0-LAYER_ADJUST_0.5-GR_OVERFLOW_0	4427.65	179202	211145	0	22:11.15	7.01316e+06	True	True	15.0	0.50
0	ibex	asap7	UTIL_45-AR_1.0-GAP_10-PD_LB_ADD_0-LAYER_ADJUST_0.5-GR_OVERFLOW_0	4502.23	117639	218727	ERR	4:22:54	5.54974e+07	True	False	45.0	0.50
0	ibex	asap7	UTIL_22-AR_1.0-GAP_10-PD_LB_ADD_0-LAYER_ADJUST_0.6-GR_OVERFLOW_0	4450	156414	220298	0	26:03.42	7.7621e+06	True	True	22.0	0.60
0	ibex	asap7	UTIL_54-AR_1.0-GAP_10-PD_LB_ADD_0-LAYER_ADJUST_0.32-GR_OVERFLOW_0	4510.35	114516	228853	ERR	1:37:06	5.56105e+07	True	False	54.0	0.32
0	jpeg	asap7	UTIL_29-AR_1.0-GAP_10-PD_LB_ADD_0-LAYER_ADJUST_0.42-GR_OVERFLOW_0	2560.06	205054	421858	0	1:11:55	1.48011e+07	True	True	29.0	0.42

• Let us examine the scatter plot of success/failures for the two platforms.

Text(0.5, 1.0, 'asap7')

• We observe that in the sky130hd platform the sweet spot for utilization is in the about 35 and the for the layer_adjust is below 0.5. For the asap7 platform both the core utilization and layer adjust can be higher. However asap7 does shw designs that are failing at lower utilization and layer adjust values.

• Print the statistics for each of the entries in the dataframe.

	count	mean	std	min	25%	50%	75%	max
GR WNS	4066.0	9.820231e+02	1.706570e+03	-711.72	-1.21	2.18	2.488752e+03	4507.34
Wire Length	4066.0	7.123667e+05	6.787812e+05	109766.00	160621.00	222730.00	1.382211e+06	2592657.00
Vias	4066.0	2.716883e+05	9.639511e+04	139823.00	204418.00	228510.00	3.117650e+05	470922.00
DRC Errors	4066.0	1.261682e-01	3.167030e+00	0.00	0.00	0.00	0.000000e+00	116.00
Peak Mem	4066.0	7.211812e+06	6.524477e+06	1223584.00	1754770.00	5961410.00	1.135811e+07	55782244.00
Core Util	4066.0	2.896950e+01	1.293604e+01	10.00	18.00	27.00	3.900000e+01	60.00
Layer Adjust	4066.0	3.892622e-01	1.167535e-01	0.20	0.28	0.38	4.800000e-01	0.60
Runtime	4066.0	1.521414e+03	1.615230e+03	191.00	457.00	671.00	2.403000e+03	16183.00

            Runtime      Peak Mem
count  4066.000000   4.066000e+03
mean   1521.413920   7.211812e+06
std    1615.229565   6.524477e+06
min    191.000000    1.223584e+06
25%    457.000000    1.754770e+06
50%    671.000000    5.961410e+06
75%    2403.000000   1.135811e+07
max    16183.000000  5.578224e+07

• We can observe a huge variation in runtime for the different runs and also a substantial variation in the routed wirelength. We would like to observe the relation between the various parameters on both runtime and routed wirelength.

Print some of base metrics from the data set.

• Number of success/failures.
• Min, Max of Wirelength, number of vias, drc errors.

Number of successful runs: 4066
Number of failed runs: 2360

Min Wire Length: 109766.0,  Max Wire Length: 2592657.0

Min Vias: 139823,  Max Vias: 470922

Min DRC Errors: 0,  Max DRC Errors: 116

• Plot the number of successful runs and number of failures with respect to core utlization. We can see that the trends are what we would expect. As utilization increases, the number of successful runs decreases and the number of failure runs increases.

[NbConvertApp] WARNING | pattern 'METRICS2.1-fine-grain.ipynb' matched no files
This application is used to convert notebook files (*.ipynb) to various other
formats.

WARNING: THE COMMANDLINE INTERFACE MAY CHANGE IN FUTURE RELEASES.

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to logging.DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=10]
--generate-config
    generate default config file
    Equivalent to: [--JupyterApp.generate_config=True]
-y
    Answer yes to any questions instead of prompting.
    Equivalent to: [--JupyterApp.answer_yes=True]
--execute
    Execute the notebook prior to export.
    Equivalent to: [--ExecutePreprocessor.enabled=True]
--allow-errors
    Continue notebook execution even if one of the cells throws an error and include the error message in the cell output (the default behaviour is to abort conversion). This flag is only relevant if '--execute' was specified, too.
    Equivalent to: [--ExecutePreprocessor.allow_errors=True]
--stdin
    read a single notebook file from stdin. Write the resulting notebook with default basename 'notebook.*'
    Equivalent to: [--NbConvertApp.from_stdin=True]
--stdout
    Write notebook output to stdout instead of files.
    Equivalent to: [--NbConvertApp.writer_class=StdoutWriter]
--inplace
    Run nbconvert in place, overwriting the existing notebook (only 
    relevant when converting to notebook format)
    Equivalent to: [--NbConvertApp.use_output_suffix=False --NbConvertApp.export_format=notebook --FilesWriter.build_directory=]
--clear-output
    Clear output of current file and save in place, 
    overwriting the existing notebook.
    Equivalent to: [--NbConvertApp.use_output_suffix=False --NbConvertApp.export_format=notebook --FilesWriter.build_directory= --ClearOutputPreprocessor.enabled=True]
--no-prompt
    Exclude input and output prompts from converted document.
    Equivalent to: [--TemplateExporter.exclude_input_prompt=True --TemplateExporter.exclude_output_prompt=True]
--no-input
    Exclude input cells and output prompts from converted document. 
    This mode is ideal for generating code-free reports.
    Equivalent to: [--TemplateExporter.exclude_output_prompt=True --TemplateExporter.exclude_input=True]
--allow-chromium-download
    Whether to allow downloading chromium if no suitable version is found on the system.
    Equivalent to: [--WebPDFExporter.allow_chromium_download=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--config=<Unicode>
    Full path of a config file.
    Default: ''
    Equivalent to: [--JupyterApp.config_file]
--to=<Unicode>
    The export format to be used, either one of the built-in formats
    ['asciidoc', 'custom', 'html', 'latex', 'markdown', 'notebook', 'pdf',
    'python', 'rst', 'script', 'slides', 'webpdf'] or a dotted object name that
    represents the import path for an `Exporter` class
    Default: ''
    Equivalent to: [--NbConvertApp.export_format]
--template=<Unicode>
    Name of the template to use
    Default: ''
    Equivalent to: [--TemplateExporter.template_name]
--template-file=<Unicode>
    Name of the template file to use
    Default: None
    Equivalent to: [--TemplateExporter.template_file]
--writer=<DottedObjectName>
    Writer class used to write the  results of the conversion
    Default: 'FilesWriter'
    Equivalent to: [--NbConvertApp.writer_class]
--post=<DottedOrNone>
    PostProcessor class used to write the results of the conversion
    Default: ''
    Equivalent to: [--NbConvertApp.postprocessor_class]
--output=<Unicode>
    overwrite base name use for output files. can only be used when converting
    one notebook at a time.
    Default: ''
    Equivalent to: [--NbConvertApp.output_base]
--output-dir=<Unicode>
    Directory to write output(s) to. Defaults to output to the directory of each
    notebook. To recover previous default behaviour (outputting to the current
    working directory) use . as the flag value.
    Default: ''
    Equivalent to: [--FilesWriter.build_directory]
--reveal-prefix=<Unicode>
    The URL prefix for reveal.js (version 3.x). This defaults to the reveal CDN,
    but can be any url pointing to a copy  of reveal.js.
    For speaker notes to work, this must be a relative path to a local  copy of
    reveal.js: e.g., "reveal.js".
    If a relative path is given, it must be a subdirectory of the current
    directory (from which the server is run).
    See the usage documentation
    (https://nbconvert.readthedocs.io/en/latest/usage.html#reveal-js-html-
    slideshow) for more details.
    Default: ''
    Equivalent to: [--SlidesExporter.reveal_url_prefix]
--nbformat=<Enum>
    The nbformat version to write. Use this to downgrade notebooks.
    Choices: any of [1, 2, 3, 4]
    Default: 4
    Equivalent to: [--NotebookExporter.nbformat_version]

Examples
--------

    The simplest way to use nbconvert is
    
    > jupyter nbconvert mynotebook.ipynb --to html
    
    Options include ['asciidoc', 'custom', 'html', 'latex', 'markdown', 'notebook', 'pdf', 'python', 'rst', 'script', 'slides', 'webpdf'].
    
    > jupyter nbconvert --to latex mynotebook.ipynb
    
    Both HTML and LaTeX support multiple output templates. LaTeX includes
    'base', 'article' and 'report'.  HTML includes 'basic' and 'full'. You
    can specify the flavor of the format used.
    
    > jupyter nbconvert --to html --template lab mynotebook.ipynb
    
    You can also pipe the output to stdout, rather than a file
    
    > jupyter nbconvert mynotebook.ipynb --stdout
    
    PDF is generated via latex
    
    > jupyter nbconvert mynotebook.ipynb --to pdf
    
    You can get (and serve) a Reveal.js-powered slideshow
    
    > jupyter nbconvert myslides.ipynb --to slides --post serve
    
    Multiple notebooks can be given at the command line in a couple of 
    different ways:
    
    > jupyter nbconvert notebook*.ipynb
    > jupyter nbconvert notebook1.ipynb notebook2.ipynb
    
    or you can specify the notebooks list in a config file, containing::
    
        c.NbConvertApp.notebooks = ["my_notebook.ipynb"]
    
    > jupyter nbconvert --config mycfg.py

To see all available configurables, use `--help-all`.